Enable CPU accelerate FIL in MULTI #7380
Conversation
openvino-pushbot
added
category: inference
tiger100256-hu
force-pushed
the
huyuan/multi_cpu_accelerator
branch
from
c5c6efa to
a83fcdb
Compare
inference-engine/src/multi_device/multi_device_exec_network.cpp
Outdated
Show resolved
Hide resolved
inference-engine/src/multi_device/multi_device_exec_network.cpp
Outdated
Show resolved
Hide resolved
inference-engine/tests/functional/plugin/cpu/shared_tests_instances/behavior/config.cpp
Outdated
Show resolved
Hide resolved
inference-engine/src/multi_device/multi_device_exec_network.cpp
Outdated
Show resolved
Hide resolved
| ScheduleToWorkerInferRequest(std::move(t), device); | ||
| } | ||
| }); | ||
| _networkFirstReady = _cpuFuture.get(); |
There was a problem hiding this comment.
we are waiting for the CPU here, but the accel may be loaded first (from the cache), let's keep this in mind (but maybe not complicate this PR anymore with that regard)
There was a problem hiding this comment.
Agree. Follow up PR is needed for LoadNetwork cancelling and accel completion first.
inference-engine/src/multi_device/multi_device_async_infer_request.cpp
Outdated
Show resolved
Hide resolved
tiger100256-hu
force-pushed
the
huyuan/multi_cpu_accelerator
branch
2 times, most recently
from
8901240 to
06565f1
Compare
inference-engine/src/multi_device/multi_device_exec_network.cpp
Outdated
Show resolved
Hide resolved
inference-engine/src/multi_device/multi_device_exec_network.cpp
Outdated
Show resolved
Hide resolved
tiger100256-hu
force-pushed
the
huyuan/multi_cpu_accelerator
branch
from
a77cbae to
a6a52b3
Compare
Signed-off-by: Hu, Yuan2 <[email protected]>
KEY_PERFORMANCE_HINT_NUM_REQUESTS Signed-off-by: Hu, Yuan2 <[email protected]>
| // and reset the strDevices to support devices | ||
| std::vector<std::string> validConfigKey; | ||
| validConfigKey.push_back(PluginConfigParams::KEY_PERF_COUNT); | ||
| validConfigKey.push_back(PluginConfigParams::KEY_EXCLUSIVE_ASYNC_REQUESTS); |
There was a problem hiding this comment.
minor comment: I assume this ASYNC_REQUESTS config key is only needed becuase OV has legacy impl and tests for that. but not sure of new devices like VPUX will support that, as the mode that is triggered by this key is obslolete.
So let's check that this checking this (evetually useless) config capability doesn't prevent us form using new devices like CUDA and KMB, and if this is the case- will need to remove maybe in the next PR
| } | ||
| // replace the configure with configure that auto want to pass to device | ||
| // and reset the strDevices to support devices | ||
| std::vector<std::string> validConfigKey; |
There was a problem hiding this comment.
minor but better to do std::vectorstd::string validConfigKey = PerfHintsConfig::SupportedKeys(); rather than explicitly add the specific values in the lines 241-242 below ( since if the new config is added to the hints impl, you will need to explicitly add that)
* Enable CPU accelerate FIL in MULTI Signed-off-by: Hu, Yuan2 <[email protected]> * add configure to device KEY_PERFORMANCE_HINT_NUM_REQUESTS Signed-off-by: Hu, Yuan2 <[email protected]>
Details:
Tickets: